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Why  Neuroergonomics? 


•  To  design  effective  human-machine  systems,  we  must 

>  Understand  mind  in  relation  to  work  and  technology — ergonomics 

>  Mind  cannot  be  understood  without  studying  the  brain — neuroscience 

>  Hence  study  brain  and  mind  in  complex  work  domains — Neuroergonomics 

•  Neuroergonomics  can  provide  for  more  effective  and  natural 

interaction  between  humans  and  technology 


Oxford  University  Press,  2008 


Two  Examples  of  Neuroergonomics 


*  Neuroimaging  and  adaptive  automation 

>  Enhancing  performance  of  operators  supervising  multiple 

unmanned  vehicles 

•  Molecular  genetics  and  proteomics 

>  Identifying  rapid  decision  makers  in  command  and  control 
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Example  1: 

Neuroimaging  and  Adaptive  Automation 


Enhancing  performance  of  operators  supervising 

multiple  unmanned  vehicles 
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Robotic  Evolution  Overview* 


1990  2010  2020 
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Shared  supervisory 
control  will  accelerate 
transition 


2005  Joint  Robotics  Program  Master  Plan 
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Soldier-Robot  Teaming 


•Unmanned  vehicles  being 
introduced  in  Army  systems  to 

>  extend  manned  capabilities 

>  provide  tactical  flexibility 

>  act  as  “force  multipliers” 

•  Goal:  Enhance  Soldier-system 
performance  while  optimizing 
workload 

•  Approach:  Use  adaptive 
automation  to  provide  support  to 
Soldier  when  and  where  needed 
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Adaptive  Automation 

An  approach  to  automation  in  which  the  “division  of  labor ”  between 


human  and  machine  is  flexible  and  context-dependent 


Optimized  user 
performance 


Adapts  Interface 
Behavior 


Adaptive  System 
Manager 


Task  Context 
■< - 


Cognitive 

Workload 

Assessment 


User  engaged 
in  a  cognitive 
task 


Operator 


Assessment 


Arch  Lab 


/ 

m  jT  GEORGE 

Mason 


Cognitive  Events 


Triggers  for  Adaptive  Automation 


•  Critical  events 

•  Mission  phase 

•  Operator  performance 

•  User  modeling 

•  Operator  neurocognitive  states  (attention,  workload, 
situation  awareness,  fatigue  etc.) 

Parasuraman  (2000),  Ergonomics. 
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Simulation  Integration  Lab  (SIL) 


•  Reconnaissance,  Surveillance,  and 
Target  Acquisition  (RSTA) 

>  With  and  without  Automatic  Target 
Recognition  (ATR)  support 


Change  Detection  Task:  Icon 
on  Situation  Map  Moves 


*  Monitor  UAV  and  UGV  assets 


*  Secondary  change  detection  task 
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Testing  the  Efficacy  of  Adaptive  Automation 

•  Manual:  no  support 

•  Static  automation:  Automatic  Target  Recognition  (ATR)  in  middle  of 
simulated  reconnaissance  mission 

•  Adaptive  automation:  Automatic  Target  Recognition  (ATR)  in  middle 
of  simulated  mission 

•  if  and  only  if  subject’s  change  detection  performance  up  to  that 
point  in  time  is  less  than  a  threshold 


Manual 


Static 


ATR  On 


Automation 

Adaptive 


Automation 

(triggered) 


ATR  On 


Adaptive 
Automation 
(not  triggered) 


Pre 


Post 


time 


Effects  of  Adaptive  Automation  on  Situation 
Awareness  (SA)  and  Workload 
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Attention  Enhances  the  Pland  N1  Event-Related 

Brain  Potential  Components 


ERPs  and  Attention 


Brain  Topography  of 
Attention  Effect 


Fu,  Greenwood,  &  Parasuraman  (2005),  Human  Brain  Mapping 
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Change  detection  probe: 
Sine  wave  grating 
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Event-Related  Brain  Potentials 
(ERPs)  to  Attended  and 
Unattended  Probes 


Wavelet  analysis 


Attended  Probe  Unattended  Probe 

ERP  to  Probe  Event 


Time  (milliseconds) 


PI  Difference  Wave  (>iV) 


Effect  of  Adaptive  Automation  on  the  PI 
Brain  Potential  Attention  Effect 
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Manual  Static  Adaptive 
Automation  Condition 


Example  1:  Conclusions 


•  Adaptive  automation  triggered  by  operator  change  detection 
performance  enhances  human  performance  in  multiple 
UAV/UGV  supervision — increased  SA  and  reduced 
workload 

•  Brain  measures  of  attention  (PI  and  N1  components  of  the 
ERP)  provide  converging  neural  evidence  for  the  efficacy  of 
adaptive  automation 

•  A  neuroergonomic  approach  to  adaptive  automation  can 
lead  to  improved  human-machine  synergy 


>  Licklider’s  (1960)  vision  of  human-computer  symbiosis? 
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Example  2: 

Molecular  Genetics  and  Proteomics 


Identifying  rapid  decision  makers  in  command  and  control 
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Identifying  Sources  of  Individual  Differences 


•  Individual  differences  reflect 

>  Development 

>  Experience 

>  Training 

>  Genetic  factors  (natural  variation) 

•  Can  molecular  genetics  help  in  understanding 

>  Normal  variation  in  cognition? 

>  Exceptional  individuals  (“cognitive  superstars”)? 
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Effects  of  Working  Memory  Load  on 
Prefrontal  Cortex  Activation 


Linear  regression 

1 - 1 - 1 

12  4  8 

Memory  Load 


Jiang,  Haxby,  Martin,  Ungerleider,  &  Parasuraman  (2000).  Science 
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Prefrontal  Cortex  Activation  and  Working 

Memory  Load 
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Can  this  broad 
range  of  individual 
differences  at  high 
load  be  explained? 


Subject  7;  The 
Odd  Man  Out 
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Genes,  DNA,  and  SNPs 


•  Human  genome:  ~  20-25,000  genes 

~  3  billion  DNA  base  pairs  (bp) 

•  The  DNA  alphabet 

>  thymine  (T) 

>  adenine  (A) 

>  guanine  (G) 

>  cytosine  (C) 

•  DNA  base  pairs  can  have  different  forms  (alleles) 

•  Allelic  variation  often  due  to  substitution  of  one 
amino  acid  for  another — single  nucleotide 
polymorphisms  (SNPs) 

e.g . ACATAGA . vs. 

. AC AC AG A . 

•  1  SNP  for  every  1 000  bp  in  unrelated  individuals 
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Candidate  SNP  Approach 

Top-Down 

Cognitive  Function 
Regional  Brain  Network 
Neurophysiology  of  Brain  Area 
Neurotransmitter  Innervation 
Neurotransmitter  Modulation 

Protein  Regulation 

Neurotransmitter  SNP 
SNP 

SNP  Databases  (e.g.,  http://www.ncbi.nlm.nih.gov/SNP/) 

Bottom-Up 
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Gene  SNPs  Associated  with  Cognition 


•  Dopaminergic/Noradrenergic  Genes 

-  DRD4 

-  DAT  1 

-  COMT 

-  DBH  V 

•  Nicotinic  Cholinergic  Genes 
- CHRNA4 
- CHRNA7 

•  Muscarinic  Cholinergic  Genes 

-  CHRM2 

•  Genes  Affecting  Neuron  Health  and  Plasticity 

-  BDNF 

-  APOE-e4 
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Greenwood  &  Parasuraman 
(2003).  Cognitive  Neuroscience 
Reviews . 


Working  Memory  and  Complex  Decision  Making 


•  Important  moderating  factor  in  many  different  cognitive 
functions — decision  making,  problem  solving,  language 
processing,  mathematical  cognition,  etc. 

•  The  dopamine  beta  hydroxylase  (DBH)  gene  product  converts 
dopamine  to  norepinephrine  in  the  brain 

•  DBH  modulation  may  be  selective  for  prefrontal  cortex  dependent 
functions,  such  as  working  memory  and  executive  function 

•  Do  individuals  with  DBH  gene  variants 

>  Have  high  working  memory  capacity? 

>  Exhibit  higher  decision  accuracy  under  time  pressure? 
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The  DBH  Gene  and  its  SNPs 


Chromosome  9 
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Working  Memory  Task 


Fixation 
1000  ms 


1-3  Target 
locations 


500  ms 


Delay 


3000  ms 


Same  or 
Different? 
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Effects  of  T  Allele  Dose  of  DBH  -1021  C/T  SN 
on  Spatial  Working  Memory 


12  3 

Memory  Load  (Number  of  Target  Dots 
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9q34  Chromosome  9 


I - 1 - 1 - I - 1 - 1 - 1 - 1 - 1  I  I - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1  i  » 

Exon  1  Exon  12 

t  ■  I - DHH - 0 - H4 - HHD 


•  What  is  the  mechanism  by  which  DBH  is 
linked  to  working  memory? 

•  Is  DBH  a  “functional”  SNP? 

•  Genes  are  only  of  interest  if  they  are 
expressed  and  influence  proteins, 
particularly  in  the  brain 

•  Cognitive  proteomics:  linking  gene- 
controlled  proteins  to  function 


rni^lORGE 

Arch  Lab  MASON 


Effects  of  DBH  on  Synaptic  Dopamine 
(DA)  and  Norepinephrine  (NE) 

•  A.  High  DBH  Level:  NE  receptors  active 

•  B.  Low  DBH  Level:  DA  receptors  active 
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Effects  of  T  Allele  Dose  of  DBH  -1021  C/T  SNF 
on  Plasma  Dopamine  B  Hydroxylase  Levels 
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T  Allele  Dose  of  -1021  C/T  SNP 
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Decision  Making  in  Command  and  Control 


Decision  making  task  performed  both  with  and  without  automated  support 


Arch  Lab 


/ 

m  jT  GEORGE 

Mason 


DECISION  TIME  (s) 


D(3H  Enzyme  Levels  and  Decision  Time 


Manual 

m/GEORGE 

Arch  Lab  MASON 


_  High  DpH 

enzyme  activity 

h  Low  DpH 

enzyme  activity 

195%  reduction  in 

decision  time 

Automation 


Subject  7:  A  “Cognitive  Superstar” 


Has  very  high  verbal  and  spatial  working  memory — 4 
standard  deviations  above  average 

Can  maintain  attention  at  1 00%  accuracy  for  2  hours 

Shows  reduced  prefrontal  fMRI  activation  and  no  increase 
with  load  in  a  working  memory  task 

What  are  S7’s  DBH  genotypes? 

S7  has  the  T/T  genotype  in  the  -1021  C/T  SNP  associated 
with  high  working  memory  and  low  DBH  enzyme  level 

Has  lowest  blood  DBH  enzyme  level  among  650  subjects 
tested  to  date 
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What  Else  Do  We  Know  About  S7? 


•  Age  26,  Male  Graduate  Student 

•  High-average  but  not  superior  IQ 

•  Good  but  not  exceptional  grades 

•  Normal  MRI  (volumetric  analysis  of 
specific  cortical  regions  not  done) 

•  NOT  an  avid  video  game  player  (cf. 
Daphne  Bavalier  studies  on  attentional 
capacity) 
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Example  2:  Conclusions 


•  DBH — a  dopaminergic/noradrenergic  gene  expressed  strongly  in 
prefrontal  cortex — associated  with  normal  individual  differences 
in  working  memory  (verbal  and  spatial) 

•  Plasma  DBH  levels  inversely  correlated  with  working  memory  and 
decision  making  performance 

•  Molecular  genetics  provides  a  new  approach  to  understanding 

>  individual  differences  in  cognition 

>  exceptional  cognitive  performance 
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Ongoing  Research 


•  Gene-gene  interactions 

>  Effects  of  cholinergic  (CHRNA4)  and  neurotrophic 
(Alzheimer  risk)  genes  (APOE)  on  attention 

>  Interactive  effects  of  nicotinic  (CHRNA4)  and  muscarinic 
(CHRM2)  genes  on  attention 

•  Gene-environment  interactions 

>  Effects  of  COMT  and  DBH  and  variable-priority  training 
on  dual-task  performance 

>  Moderating  influence  of  COMT  and  BDNF  genes  on 
effects  of  aerobic  exercise  on  executive  attention  in  older 
adults 
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Ongoing  Research 


•  Spatial  working  memory  in  normal  and  DBH  — 
-1021T/C  knockout  mice 

•  RNA  interference  studies  in  rat  model  of  aging 
(Fischer  344  strain,  Bizon  group) 
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Neuroergonomics:  Conclusions 

•  Neuroscience  is  not  a  panacea  to  the  challenges  facing  the  Army 

>  but  appropriately  applied  neuroscience 

>  that  goes  beyond  the  bench  to  examine  complex  cognitive  functions  of 
humans  performing  real  work  in  real  settings — Neuroergonomics 

>  can  yield  great  benefits  in  enhancing  soldier  and  system  performance 

•  Two  examples  of  successful  neuroergonomics  research 

>  Neuroimaging  and  adaptive  automation 

>  Molecular  genetics  and  proteomics 

•  Neuroergonomics  can  lead  to  more  effective  and  natural 

interaction  between  humans  and  technology 
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